|
|
 |
|
Poster:
|
 tracey pooh |
Date:
|
December 22, 2009 09:46:58am |
|
Forum:
|
forums
|
Subject:
|
Re: Checking out items via FTP? |
we don't work superwell with 1000s of files just so you know.
you'll likely end up with some pretty slow items.
but let's say you want all those files anyway in *1* item.
just use the FTP uploader for the new item.
if you have to change 1000s of files in the item later, the HTTP editing is the old way -- and BTW "flash editor/uploader" ia a wrong misnomer -- people are able to opt to not use flash and instead use normal HTTP POST (ie: a normal form-based method) if they don't want to use the flash widget that monitors file progress.
however, admittedly, doing 1000s of http edits is going to be excruciating.
as a site, we're not designed for such kind of extreme usage. we work with it, but obviously it's not going to be easy for the user/uploader.
a final possible option is S3, but requires more advanced tech ability and command line for the the uploader.
can you post an example item where you have 1000s of files?
--tracey
|
Poster:
|
h4ck3rm1k3 |
Date:
|
December 22, 2009 09:51:56am |
|
Forum:
|
forums
|
Subject:
|
Re: Checking out items via FTP? |
Sure, here is the mail I sent :
Hi there,
I have started to produce openstreetmap files for the archive.
For example converted data from the EPA and census buro.
http://www.archive.org/details/ZCTA_NJhttp://www.archive.org/details/OpenstreetmapZipcodeshttp://www.archive.org/details/EPADatapointsOpenStreetmapMy problem is that I have lots of small files, and would like to
update them with FTP.
Also, what I really need is some form of GIT/BZR/HG distributed
version control system for editing these files.
they will be updated on a regular basis.
The data is publicdomain/creative commons, but archive.org is not
ideal for processing them.
Are you interested in working with me to making a better way to host
map data on archive org?
I am a programmer and would like to work with you on this.
thanks,
mike
|
Poster:
|
h4ck3rm1k3 |
Date:
|
December 22, 2009 09:56:35am |
|
Forum:
|
forums
|
Subject:
|
Re: Checking out items via FTP? |
The other thing that I would like to say is that for these zip code files we need to have small individual files, because people will only access one or two at a given time.
To put them all into a zip file would defeat the purpose, unless there is a way to access only a certain offset in the file some how.
|
Poster:
|
tracey pooh |
Date:
|
February 18, 2010 02:01:31pm |
|
Forum:
|
forums
|
Subject:
|
Re: Checking out items via FTP? |
the only robo option for something like that that we have available is S3.
that would allow for all of what you want i think..
-t
|
Poster:
|
tracey pooh |
Date:
|
February 18, 2010 02:02:25pm |
|
Forum:
|
forums
|
Subject:
|
Re: Checking out items via FTP? |
and once again, lots of small files = fail with our site.
we have some interesting tools/solutions for that,
including an automatic and efficient 'zipview' utility
that we could try to setup for your items.
this allows a single file to be pulled out a of zip to an end user.
--t
TVNews
OpenLibrary